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EXECUTIVE  SUMMARY 


This  report  is  the  second  in  a  three-part  series  that  describes  a  research  program  to  improve  the 
selection  and  retention  of  airline  passenger  pre-board  screener  personnel.  This  document  details 
the  methodology  used  to  conduct  a  job  task  analysis  of  the  X-ray  position.  The  job  task  analysis 
was  conducted  to  provide  guidance  in  the  selection  and  development  of  potential  predictor 
instruments. 

The  reader  is  directed  to  the  first  report  for  an  examination  of  the  literature  related  to  the 
selection  and  performance  measurement  issues.  Issues  related  to  job  tenure,  job  satisfiers  and 
dissatisfiers,  and  recommendations  to  reduce  employee  turnover  can  be  found  in  the  final  report 
of  the  series. 
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1. 


INTRODUCTION. 


A  widely  accepted  technique  in  use  today  to  determine  the  aptitude  dimensions  that  are  needed 
within  a  valid  selection  system  are  to  conduct  a  thorough  job  task  analysis  (JTA).  The  task 
analysis  should  be  complete  enough  to  provide  sufficient  detail  as  to  all  task  elements  and 
subtasks  required  of  the  job.  Task  elements  should  define  all  actions,  decisions,  receipts  and 
requests  for  information,  and  communication  links  involved  within  the  position.  The  level  of 
specification  should  permit  an  evaluation  of  the  information  processing,  cognitive,  perceptual, 
and  psychomotor  abilities  required  to  perform  each  task  element.  Identification  of  the  requisite 
information  processing  abilities  provides  the  foundation  of  appropriate  selection  models. 

A  task  analysis  however,  while  defining  all  task  elements,  does  not  necessarily  describe  a  set 
order  of  functions  that  all  operators  rigidly  engage.  A  task  analysis  does  not  imply  that  all 
operators  perform  all  the  indicated  task  elements.  The  task  analysis  simply  represents  all  given 
task  elements  that  are  possible  functions  of  a  specific  job.  This  is  particularly  pertinent  with 
respect  to  the  X-ray  scanning  position  for  airline  passenger  security  screeners.  The  image 
displayed  on  the  monitor  is  a  complex  visual  field  that  can  be  analyzed  and  interpreted  using 
several  visual  characteristics.  For  example,  aspects  of  the  visual  field  may  be  analyzed  for  visual 
characteristics  defined  by  size,  shape,  or  density.  The  decision  reached  by  the  operator  on  the 
next  course  of  action  is  dependent  upon  whether  certain  criteria  are  met  or  not  met  for  that 
characteristic.  The  objective  of  conducting  this  JTA  is  to  identify  all  task  elements  and  decisions 
that  are  required  for  successful  target  detection. 

The  number  of  characteristics  that  are  analyzed  by  screeners  may  vary  considerably  in  detecting 
targets.  The  amount  of  information  used  by  a  screener  may  well  define  the  effectiveness  or 
quality  of  performance  for  the  X-ray  scanning  position.  The  focus  of  the  analysis  is  to  determine 
what  visual  characteristics,  or  types  of  information,  are  used  by  screeners  while  on  task.  The 
analysis  was  also  conducted  to  determine  if  there  are  differences  among  screeners  in  the  type  or 
quantity  of  information  used  in  analyzing  each  image.  We  also  sought  to  determine  how  critical 
each  display  characteristic  was  in  the  screening  process.  Several  facilities  have  been  included  in 
the  analysis  to  ensure  generalizability  with  regard  to  equipment,  security  contractor,  and 
geographic  area.  A  secondary  goal  of  the  task  analysis  was  to  examine  the  relationship  between 
experience  and  the  type  of  information  used  in  screening  all  items  entering  the  sterile  area. 

2  METHODOLOGY  AND  RESULTS. 

2.1  PHASE!. 

Phase  I  of  the  JTA  was  conducted  at  Orlando  International  Airport  (MCO)  to  develop  a 
functional  flow  analysis  of  the  X-ray  scanning  position.  The  flow  analysis  was  undertaken  to 
identify  all  task  elements,  decision  points,  actions,  and  activities  related  to  the  position.  An 
operations-decision  approach  (Jonassen,  Hannum,  and  Tessma,  1989;  Meister,  1989)  was  used  to 
complete  this  work. 
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The  functional  flow  analysis  data  was  generated  using  a  number  of  techniques;  (a)  training,  (b) 
observation,  (c)  incumbent  interviews,  (d)  supervisor  and  instructor  interviews,  and  (e)  hands-on 
operation  of  the  X-ray  equipment.  This  methodology  allowed  for  ensuring  that  all  task  elements 
of  the  position  were  discretely  identified.  Utilization  of  these  techniques  also  permitted 
sequencing  the  required  functions  of  the  screening  process. 

Three  investigators  completed  the  FAA/ATA  airline  passenger  security  screening  curriculum  to 
formal  certification.  "Two  investigators  continued  to  “work  the  position”  beyond  certification  as 
security  team  members.  During  the  training  and  certification  process  all  elements  of  the  X-ray 
screening  task  were  demonstrated  and  practiced.  This  gave  the  research  investigators  an 
opportunity  to  identify  task  elements  in  a  structured  training  environment.  In  addition,  it  was 
possible  to  isolate  many  of  the  decision  points  involved  in  the  process,  while  acquiring  an 
understanding  of  the  X-ray  display  characteristics  that  had  to  be  identified  and  analyzed.  The 
training  and  certification  process  required  7-8  days  to  complete  (including  mandated  State  of 
Florida  Class  D  Security  Officer  Certification). 


In  conjunction  with  training,  screeners  were  observed  and  interviewed  on-line  to  identify 
additional  cues,  procedures,  actions,  decisions,  and  display  characteristics  that  were  used  while 
on  position.  This  process  further  extended  the  breath  of  the  functional  flow  analysis  and  offered 
an  environment  to  identify  strategies  used  in  the  screening  process.  Observing  and  interviewing 
screeners  also  provided  further  structure  to  the  analysis  by  defining  the  typical  flow  of  activities 
and  actions  that  were  required  in  the  operational  environment.  Extensions  and  additions  to  the 
flow  analysis  that  were  obtained  from  screeners  were  verified  with  supervisory  and  instructional 
personnel.  In  most  cases  these  additional  task  elements  were  not  included  in  the  training 
program  and-were  expected  to  be  acquired  through  job  experience. 

The  completion  of  Phase  I  resulted  in  the  construction  of  the  functional  flow  diagram  shown  in 
appendix  A.  This  diagram  depicts  all  possible  activities  and  decision  chains  for  the  X-ray 
screening  position  during  the  normal  work  shift.  While  clearly  defining  the  task  elements 
required  for  target  detection  and  screening,  it  does  not  represent  the  position  as  performed  by  all 
personnel.  The  X-ray  display  characteristics  that  are  examined,  and  the  alternative  actions 
chosen  by  each  screener  are  partly  individualized.  The  functional  flow  diagram  only  captures  all 
probable  task  elements  that  are  inherent  in  the  position  and  offer  a  working  model  for  the  task 
analysis. 

2.2  PHASE  n. 

This  component  of  the  JTA  was  conducted  to  determine:  (a)  if  the  functional  flow  analysis  was 
complete  and  accurate;  (b)  if  the  structure  of  the  flow  analysis  was  generalizable  to  airport 
checkpoints  operated  by  a  different  security  firm;  (c)  the  number  of  X-ray  display  characteristics 
used  by  screeners  in  analyzing  carry-on  items;  and  (d)  the  scan  patterns  used  by  screeners. 

All  Phase  n  work  was  carried  out  at  Fort  Lauderdale  International  Airport  (FLL)  over  a  two-day 
period.  Investigators  observed  and  interviewed -screeners  on-line  at  two  different  security 
checkpoints.  Three  work  shifts  were  observed.  Most  of  the  screeners  were  observed  for  two  30- 


2 


minute  shifts.  In  all,  15  screeners  (all  available  personnel  working  the  X-ray  position  during  the 
site  visit)  were  observed  and  interviewed.  Job  longevity  ranged  from  3  to  75  months  with  a 
mean  time  on  job  of  26.9  months  (Median  =  13.9  months).  Job  longevity  for  this  site  was 
uncharacteristically  long  as  compared  to  the  rest  of  the  industry  (note  that  this  is  not  a 
representative  sample  of  the  national  workforce). 

A  simple  data  collection  instrument  was  developed  to  assist  with  the  observation  and  interview 
process  (see  appendix  B).  Items  within  the  instrument  were  not  presented  verbatim  to  the 
screeners  but  were  instead  queried  in  reference  to  specific  visual  characteristics  of  the  X-ray 
display.  For  example,  if  a  specific  item  or  area  caused  the  screener  to  examine  the  carry-on 
closely  (e.g.,  stop  the  belt,  call  for  a  bag  check,  confer  with  a  supervisor,  have  the  carry-on  re¬ 
entered  for  a  second  pass  through  the  X-ray  devices,  etc.),  the  investigators  questioned  the 
screener  as  to  the  characteristics  of  the  image  that  caused  concern  (i.e.,  size,  shape,  density). 
During  the  interview  the  investigators  also  queried  the  screener  as  to  the  importance  or 
significance  of  those  characteristics  in  identifying  targets  or  the  items  in  question.  The 
evaluative  component  was  limited  to  a  high,  medium,  or  low  assessment.  This  same  approach 
was  also  used  when  the  screeners  passed  carry-ons  through  the  X-ray  machine  without  incident. 
In  these  instances  screeners  were  queried  what  components  or  characteristics  were  missing  that 
resulted  in  a  “pass”  decision. 

A  secondary  objective  of  this  phase  was  to  determine  if  screeners  used  a  standardized  scan 
pattern  when  viewing  the  X-ray  display.  This  was  typically  done  while  an  image  remained  on  the 
display,  but  while  no  further  carry-ons  were  entering  the  X-ray  device.  Screeners  were  asked  to 
demonstrate  their  scan  pattern  by  tracing  the  pattern  with  their  hand  on  the  display.  Investigators 
would  verify  these  responses  by  observing  the  eye  movements  during  further  screening 
operations. 

Regarding  the  nature  of  the  scan  patterns,  we  found  that  screeners  develop  scans  that  are  highly 
individualized.  Although  the  scan  patterns  were  individualized,  two  major  techniques  were 
apparent.  Some  screeners  would  scan  from  the  entry  side  to  the  exit  side  of  the  display  (right  to 
left  or  left  to  right  dependent  on  the  position  of  the  checkpoint  with  reference  to  the  non-sterile 
passenger  area)  while  scanning  from  top  to  bottom.  The  second  major  scan  pattern  would  move 
from  entry  to  exit  side,  but  instead  alternated  with  directed  attention  to  darker  areas  of  the  image. 
Scan  patterns  were  not  taught  as  part  of  the  training  regime. 

The  functional  flow  analysis  developed  from  the  MCO-based  data  accurately  and  completely 
described  the  X-ray  screener  position  at  the  FLL  security  checkpoints.  In  fact,  no  additional  task 
elements  or  task  characteristics  emerged.  Consequently  the  working  model  appeared  valid  and 
no  revision  was  required. 

The  most  important  facet  of  this  phase  was  to  ascertain  the  relative  importance,  and  distribution 
of  use,  of  X-ray  display  characteristics  among  screeners.  This  phase  of  the  work  was  also  used  to 
refine  the  interview  process  and  re-develop  data  collection  instruments  for  the  follow-on  phases 
of  the  task  analysis.  Our  findings  indicated  that  the  most  important  display  characteristics  for 
identifying  carry-on  contents  were:  (a)  size  of  items,  (b)  the  size  of  opaque  areas  that  might 
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mask  other  contents,  (c)  shading  or  impenetrability  of  the  X-ray  (represented  by  color),  (d) 
presence  of  components  that  might  be  assembled  to  construct  incendiary  or  explosive  devices, 
and  (e)  shape  or  form  of  present  images.  Of  less  significance  were:  (a)  angle  objects  appear 
(only  40  percent  of  the  sample  indicated  they  mentally  rotate  questionable  objects  to  reach 
decisions);  (b)  degree  of  clutter  or  amount  of  material  in  the  carry-on;  and  (c)  presence  of  items 
such  as  photographic  equipment,  computers,  flashlights,  toy  weapons,  and  scissors.  Knives  and 
tools  were  scrutinized  dependent  on  quantity  and  size. 

These  data  warranted  the  examination  of  display  features  used  by  screeners  at  other  security 
checkpoint  facilities.  Although  the  Phase  n  work  identified  all  the  display  characteristics  used  by 
screeners  in  identifying  targets,  it  did  not  clearly  show  the  differences  among  screeners  with 
regard  to  the  number  of  parameters  used  in  the  process.  It  was  primarily  instrumental  in 
assessing  the  importance  of  various  display  characteristics.  It  is  possible  that  the  number  of 
characteristics  employed  in  the  screening  process  reflects  the  depth  of  analysis  conducted.  We 
believe  the  depth  of  analysis  used,  as  reflected  by  the  number  of  characteristics  analyzed,  may 
provide  a  measure  of  the  effectiveness  of  screener  performance. 

2.3  PHASE  m. 

Phase  EH  was  conducted  solely  to  determine  if  differences  exist  among  screeners  with  respect  to 
the  number  of  X-ray  display  characteristics  used  in  the  screening  process.  The  objective  of  this 
work  was  to  assess  if  screeners  are  different  in  the  depth  of  analysis  they  conduct  while 
screening.  These  data  provide  a  measure  of  how  representative  the  functional  flow  analysis  is  in 
describing  the  depth  of  analysis  portion  of  the  X-ray  screening  task. 

The  work  was  carried  out  at  six  security  checkpoints  at  Newark  International  Airport  (EWR)  and 
New  York’s  John  F.  Kennedy  International  Airport  (JFK)  over  a  four-day  period.  EWR  was 
selected  upon  recommendation  of  the  primary  security  contractor  as  representing  a  facility  where 
substantial  differences  among  screeners  could  be  observed  in  screening  performance.  This 
facility,  as  a  result  of  vast  differences  in  the  various  checkpoints  with  regard  to  task  difficulty 
level,  assigns  screeners  based  on  observed  job  performance.  In  short,  the  most  capable  screeners 
are  assigned  to  the  most  difficult  concourses  generally  characterized  by  the  amount  of  passenger 
activity  and  other  peripheral  problems  (e.g.,  baggage  theft).  This  management  technique  also 
provided  an  opportunity  to  discuss  with  supervisory  personnel  those  performance  elements  that 
were  used  to  distinguish  screeners.  JFK  represents  an  additional  facility  in  the  same  metropolitan 
area,  but  operated  by  a  different  security  organization.  This  afforded  the  opportunity  to  compare 
screeners  from  two  organizations  within  the  same  geographic  area  at  comparable  airports. 

As  previously  done,  a  simple  data  collection  instrument,  based  on  the  functional  flow  analysis 
and  the  work  carried  out  at  FLL,  was  used  to  record  job  performance.  The  instrument  was 
basically  dichotomous  in  nature,  allowing  investigators  only  to  indicate  whether  or  not  a  screener 
used  a  display  dimension  in  screening  carry-ons.  The  instrument  is  presented  in  appendix  B. 
Three  investigators,  all  trained  in  the  screening  process,  conducted  the  work.  Investigators  would 
observe  and  interview  incumbents  on-line.  Supervisory  personnel  were  interviewed  off-line  to 
determine  the  performance  elements  used  to  evaluate  screener  personnel. 
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Eighty-one  (81)  airline  passenger  security  screeners  operating  the  X-ray  position  were  observed. 
The  job  longevity  of  this  sample  ranged  from  2-72  months  with  an  average  time  on  job  of  19.8 
months  (EWR:  M  =  14.1,  s  =  12.3;  JFK;  M  =  25.8,  s  =  22.7).  Of  the  total  sample  (N  =  81),  32 
screeners  were  male  (40  percent)  and  49  were  female  (60  percent). 

Phase  in  data  are  reported  by  perceived  order  of  importance  in  table  1  below.  There  were  no 
significant  differences  found  between  the  two  airports.  The  data  clearly  demonstrate  that  there 
are  individual  differences  among  screeners  with  regard  to  the  number  of  display  characteristics 
used  in  the  screening  process.  In  particular,  a  sub-sample  of  the  population  tends  to  use  all 
available  information  in  the  screening  process.  We  also  found  that  some  screeners  would  rely 
only  on  a  few  display  characteristics  while  on  position.  These  individuals  clearly  have  a  narrow 
range  of  perceptual  features  incorporated  into  their  scan  pattern,  which  substantially  reduces  the 
level  of  analysis  conducted  on  each  item. 


TABLE  1.  RESPONSES  OBTAINED  FROM  INTERVIEWS  WITH  SECURITY  SCREENERS 

(N  =  81) 


Number  of  Responses 

Display  Dimension(s) 

Yes  ( percent) 

No  ( percent) 

Impenetrability  of  the  object  (density) 

79  (98%) 

2  (2%) 

Shape  or  Form  of  the  image 

76  (94%) 

5  (6%) 

Large  area  of  Opaque  Shading  or  Masking 

71  (90%) 

8  (10%) 

Size  of  questionable  area 

65  (80%) 

16  (20%) 

Presence  of  Possible  Components  (wires, 
batteries,  clocks,  bottles) 

59  (74%) 

21  (26%) 

Profile  or  Angle  of  object: 

Is  it  important  to  see  the  object 
from  the  side,  top,  or  other  angle? 

59  (74%) 

21  (26%) 

Do  you  mentally  rotate  the  object 
when  necessary? 

43  (54%) 

37  (46%) 

Amount  of  Clutter  in  bag 

52  (66%) 

27  (34%) 

Do  any  of  the  following  require  a  bag  check? 

Toy  Guns 

Computers 

Flashlights 

75(100%) 

31  (44%) 

16  (24%) 

0 

40  (56%) 

50  (76%) 

Any  other  factors  you  look  for  not 
mentioned? 

4(14%) 

24  (86%) 

* 
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3. 


DISCUSSION. 


In  conducting  the  JTA  and  developing  the  functional  flow  analysis,  96  screeners  were  observed 
in  the  performance  of  the  X-ray  screening  process.  Performance  differences  among  screeners, 
within  and  between  facilities,  were  striking.  Particularly  outstanding  performance  was  observed 
among  some  screeners  who  were  adept  at  identifying  both  test  targets  (FAA,  security  firm, 
airline)  and  unplanned  possible  threat  items  (passenger  carried).  Our  identification  of 
outstanding  screeners  was  consistent  with  checkpoint  security  supervisor  and  station  manager 
evaluations. 

In  one  specific  incident  a  screener  had  detected  the  presence  of  an  opened  bottle,  several 
batteries,  a  timing  device,  and  wiring  not  associated  with  the  internal  components  of  electronic 
devices.  These  items,  taken  together,  could  conceivably  represent  components  of  an  incendiary 
device.  Recognition  of  this  carry-on  as  a  possible  threat  required  substantial  depth  in  cognitive 
processing,  numerous  decisions,  isolation  and  identification  of  key  components  in  a  complex 
visual  field,  and  understanding  the  dynamics  of  threatening  devices.  Consistent  with  the  model 
developed  from  the  functional  flow  analysis,  this  individual  analyzed  the  contents  of  the  carry-on 
along  several  domains.  This  incident  was  characteristic  of  many  such  “bag  checks”  that  resulted 
in  identifying  non-permissible  items. 

Screeners  who  performed  the  X-ray  screening  process  at  a  high  level  of  competency  were  found 
to  analyze  an  extensive  array  of  display  characteristics.  In  other  words,  superior  performers  were 
representative  of  the  decision  process  depicted  in  the  functional  flow  analysis.  In  contrast, 
screeners  who  were  marginal  performers  lacked  a  comprehensive  decision  process.  The  less 
proficient  screeners  would  characteristically  analyze  X-ray  images  using  a  limited  number  of 
display  characteristics.  Generally,  such  performance  was  characterized  by  requesting  bag  checks 
only  when  an  object  was  impenetrable  and/or  the  shape  was  similar  to  one  of  the  FAA  test  items. 
Although  this  limited  analysis  of  the  X-ray  image  was  sufficient  for  passing  airline,  security  firm 
and  FAA  checks,  it  was  not  adequate  to  identify  actual  potential  threats.  It  appears  that  these  less 
proficient  screeners  only  look  for  the  FAA  test  items  and  learn  how  to  recognize  them  through 
repetition.  Actual  potential  threats  may  not  match  the  shape  of  FAA  test  items,  nor  may  they  be 
detected  on  the  basis  of  X-ray  impenetrability  alone.  Effective  security  screening  relies  on  the 
identification  and  analysis  of  carry-on  contents,  not  pattern  recognition  of  a  few  test  items. 

The  depth  of  the  analysis,  or  number  of  levels  that  decisions  are  made  during  the  screening 
process,  appears  to  be  the  single-most  important  criteria  in  discriminating  screener  performance. 
This  is  not  to  imply  that  trait  factors  (i.e.,  motivation,  teamwork,  personality  factors)  do  not 
influence  performance.  The  results  from  the  task  analysis  and  observations  however  indicate  that 
screener  performance  is  related  to  ability  in  the  decision-making  process.  Vigilance  alone,  as  an 
ability,  is  therefore  not  likely  to  predict  screener  performance.  Although  vigilance  is  required  to 
perceive  the  relevant  inputs,  clearly  some  level  of  information  processing  is  necessary  to  identify 
targets.  Display  characteristics  have  to  be  analyzed  along  several  domains  for  effective 
identification. 
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The  extensiveness  of  the  decision-making  process,  as  represented  by  the  functional  flow  analysis 
diagram,  can  be  observed  in  relationship  to  time.  This  is  particularly  true  for  screeners  who 
perform  the  task  efficiently.  Display  images  that  are  “clean”  and  featureless  (e.g.,  garment  bag 
containing  only  cloth  items),  are  quickly  screened  and  almost  never  require  stopping  the  belt  for 
closer  analysis.  The  screening  process  is  comparatively  fast  as  no  display  characteristics  are 
present  as  input  to  the  decision  process.  Differences  in  performance  between  effective  and 
marginal  screeners  are  not  discernible  with  such  images  as  the  decision-making  process  is  only 
made  along  one  level  (impenetrability  of  X-ray). 

There  were  however,  numerous  incidents  observed  when  the  screeners  would  begin  the  image 
analysis  and  detect  a  potential  threat.  It  was  observed  and  verified  by  questioning  that  the 
screener  could  not  identify  the  contents  displayed  on  the  X-ray  image,  or  the  image  was  opaque, 
or  was  too  difficult  to  “read.”  The  screeners  recognized  that  a  potential  threat  could  exist  but 
failed  to  call  a  bag  check.  Several  occasions  were  observed  when  a  laptop  computer,  portable 
telecommunications  device,  or  a  large  flashlight  (all  examples  of  possible  threats)  were  passed 
through  and  the  screener  did  not  initiate  the  required  bag  check.  The  reason(s)  for  the  screeners 
lack  of  appropriate  actions  are  not  clear,  however  there  was  one  plausible  factor  that  may  have 
influenced  their  judgment.  When  screeners  called  for  a  bag  check  during  peak  traffic  times,  and 
that  request  resulted  in  a  false  alarm,  the  supervisor  on  duty  often  became  untated  and  expressed 
their  disappointment  with  the  action  taken.  One  sf)ecific  example  was  observed  where  the 
screener  recognized  a  potential  threat  in  a  carry-on  bag.  The  bag  clearly  had  an  opaque  item  that 
required  further  investigation  because  it  could  have  easily  contained  a  potentially  dangerous 
object.  The  screener  called  the  supervisor  over  to  administer  a  bag  check.  The  supervisor  briefly 
scanned  the  X-rayed  image  and  replied  with  a  sour  expression,  “Come  on,  that  does  not  need  to 
be  checked.”  Such  management  practices  discouraged  screeners  from  taking  the  initiative  to 
examine  all  potential  threats. 

As  the  complexity  of  the  display  image  increases,  the  amount  of  time  required  to  analyze  the 
image  consequently  increases.  This  is  expected  since  the  number  of  objects  within  the  display 
increases.  However,  based  on  the  on-line  interviews  with  screeners,  the  more  efficient  screeners 
will  also  evaluate  the  contents  along  a  greater  number  of  display  characteristics.  This 
consequently  results  in  longer  time  durations  in  evaluating  the  image  and  rendering  a  decision. 
Superior  screeners  conduct  this  level  of  analysis  to  identify  any  contents  that  may  potentially 
constitute  a  threat. 


The  relationship  between  image  complexity  and  time  was  not  as  evident  with  less  proficient 
screeners.  Moderately  and  extremely  complex  images  would  require  more  time  than  featureless 
images,  but  were  processed  in  a  shorter  time  in  comparison  to  superior  screeners.  On-line 
interviews  indicated  that  such  screeners  would  only  evaluate  the  image  along  a  few  display 
characteristics.  The  decision  process  in  many  cases  would  only  evaluate  the  characteristics  of 
impenetrability,  size,  and  shape.  Unlike  the  performance  of  superior  screeners,  less  proficient 
screeners  were  less  capable  of  identifying  the  majority  of  contents  in  an  item  being  X-rayed. 

A  limited  decision  model  among  less  proficient  screeners  was  also  observed  to  impact  the 
number  of  bag  checks.  Less  proficient  screeners  were  not  capable  of  identifying  objects  since 
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most  of  their  analysis  included  only  features  of  impenetrability  or  shape.  This  limited  analysis, 
and  the  resulting  incapability  of  identifying  objects  from  an  X-ray  image,  consequently  caused  an 
increased  number  of  bag  checks  or  Checkpoint  Security  Supervisor  (CSS)  approval  to  “clear”  the 
bags.  In  one  particular  case  this  initiated  a  bag  check  on  over  50  percent  of  the  carry-ons.  The 
screener  focused  exclusively  on  the  feature  of  impenetrability.  Any  carry-on  that  contained  items 
the  X-ray  could  not  penetrate  generated  a  bag  check.  The  vast  majority  of  these  bag  checks  were 
unnecessary  had  a  further  analysis  been  conducted.  To  illustrate,  many  bag  checks  could  have 
been  eliminated  had  display  features  of  shape  and  size  been  included  in  the  screener’ s  decision 
model.  Several  carry-ons  were  bag-checked  simply  because  of  the  presence  of  a  few  coins. 

Using  a  limited  decision  model  could  possibly  represent  a  strategy  to  reduce  task  demands  to  a 
manageable  level.  This  approach  diminishes  the  efficiency  of  the  screening  process  while  not 
making  available  the  full  capability  of  the  X-ray  hardware.  Effective  screening  requires 
perceiving  all  inputs  made  available  by  the  existing  hardware  and  basing  decisions  on  the  fullest 
range  of  information.  Some  security  screening  personnel  may  not  have  the  ability  to  process 
information  to  the  level  required  within  the  time  constraints  imposed  by  the  equipment. 

Perhaps  the  number  of  display  characteristics  used  in  analyzing  X-ray  images  is  directly  related 
to  the  perceptual  speed  of  the  screener.  As  previously  pointed  out,  the  screening  process  occurs 
at  a  relatively  constant  speed  predicated  by  the  equipment.  Those  individuals  who  can  very 
rapidly  scan  a  visual  image  and  compare  it  to  specific  mental  references  possess  the  capacity  to 
incorporate  a  more  extensive  array  of  display  characteristics.  Screeners  that  are  not  as  proficient 
with  regards  to  perceptual  speed  may  compensate  by  reducing  the  amount  of  information  scanned 
from  each  display  image.  As  such,  perceptual  speed  may  impact  target  detection  performance  by 
modulating  the  amount  and  depth  of  information  extracted  by  screeners  in  scanning  X-ray 
images.  Strong  consideration  should  be  directed  at  marker  tests  for  the  assessment  of  perceptual 
speed  as  a  potential  predictive  tool. 

The  discussion  of  airline  passenger  security  screener  abilities  has  focused  thus  far  on  the 
information  processing  abilities  after  an  image  has  been  perceived.  The  JTA  clearly  highlighted 
differences  among  screeners  with  regard  to  the  decision  models  used.  However,  equally 
important  are  the  cognitive/perceptual  abilities  required  in  X-ray  security  screening.  Possible 
directions  for  the  evaluation  of  individual  differences  that  may  relate  to  screener  performance 
include:  (a)  vigilance  measures,  (b)  pattern  recognition  and  related  constructs,  and  (c) 
disembedding  figure  from  background  perception. 

Vigilance,  as  a  theoretical  construct,  has  been  presented  in  detail  in  the  introduction  with  findings 
from  experimental  studies.  Our  purpose  here  is  to  explore  the  applications  of  laboratory  findings 
to  the  work  environment.  In  the  airport  security  checkpoint  environment  the  application  of 
vigilance  is  of  concern  in  two  respects.  Maintenance  of  vigilance  on  the  display  monitor  is 
required  to  ensure  that  no  carry-on  passes  without  inspection.  A  high  degree  of  vigilance  is 
further  required  for  each  carry-on  to  conduct  a  threat  analysis.  Maintenance  of  vigilance  in  this 
low  signal  to  noise  environment,  under  varying  conditions  of  ambient  noise,  temperature,  and 
lighting,  presents  a  highly  demanding  task.  The  workload  is  farther  complicated  by  not  having  a 
temporal  expectancy  of  the  signal,  and  by  the  range  of  signals  possible.  This  environment 
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requires  that  an  individual  is  capable  of  maintaining  a  high  degree  of  vigilance  while 
disregarding  irrelevant  environmental  distracters.  As  previously  stated  however,  simply 
maintaining  vigilance  is  necessary  only  to  the  extent  of  perception.  The  inputs  perceived  from  an 
image  still  require  processing  to  determine  the  presence  of  threats.  From  a  selection  prospective, 
any  valid  selection  battery  will  require  assessing  an  individual’s  vigilance  ability  under  high 
workload  and  distracting  conditions. 

An  integral  component  of  the  workload  in  security  screening  is  the  time  demand  component.  The 
detection  and  analysis  of  threats  occur  in  a  dynamic  environment  where  the  workload  is 
dependent  upon  passenger  flow  through  the  checkpoint.  This  creates  an  environment  where  the 
number  of  carry-ons  per  unit  time  varies  widely.  Of  primary  concern  are  those  situations  where 
there  is  a  constant  demand  for  screening.  Such  circumstances  require  screener  personnel  to 
acquire  inputs  from  a  moving  visual  field,  perceive  and  isolate  relevant  inputs,  and  quickly 
choose  an  appropriate  action.  This  continuous  cycle  of  perception  and  information  processing 
again  highlights  the  need  to  select  individuals  capable  of  sustaining  vigilance  under  high  and 
varying  workload  conditions. 

The  tasks  involved  in  X-ray  security  screening  procedures  also  warrants  the  consideration  of  an 
assessment  of  pattern  recognition.  Successful  detection  of  threats,  and  efficiency  in  the  screening 
of  carry-ons,  is  predicated  on  the  need  to  rapidly  categorize  objects  as  constituting  threats,  non¬ 
threatening  items,  and  hazardous  materials.  The  speed  and  efficiency  of  X-ray  screening  are 
obviously  enhanced  by  recognizing  familiar  patterns  that  do  not  constitute  items  of  concern  and 
eliminating  them  from  further  consideration. 

Highly  competent  screeners  were  noted  to  identify  non-targets  in  each  image  rapidly  and  easily. 
Most  of  these  identifications  were  achieved  by  simple  pattern  recognition.  Pattern  recognition  in 
this  environment  was  characterized  by  identifying  objects  from  different  perspectives  and  in 
varying  rotations.  During  most  of  the  on-line  observations/interviews  screeners  were  queried  as 
to  the  identity  of  specific  objects  and  the  angle  of  view  or  rotation  the  object  was  seen  from.  The 
capability  to  perform  this  operation  requires  understanding  the  perspective  generated  by  the 
individual  X-ray  device  (image  depiction  differs  among  devices),  and  then  mentally  generating 
an  object’s  position  relative  to  its  position  in  the  carry-on.  Pattern  recognition  is  required  for 
both  target  and  non-threat  item  detection. 

Our  analysis  indicated  that  shape  was  a  primary  display  feature  used  by  nearly  all  screeners.  The 
emphasis  on  pattern  recognition  was  also  evident  in  training  video  tapes,  on-line  training,  and 
security  firm  initiated  checks.  An  important  technique  used  in  X-ray  security  screening  was  to 
familiarize  screeners  with  the  shapes  of  potential  threats  (i.e.,  FAA  targets)  from  several 
perspectives  with  the  expectation  of  recognizing  actual  targets  when  they  occur.  The  underlying 
assumption  was  that  threats  can  be  recognized  by  training  personnel  to  respond  to  one  specific 
target,  with  the  expectation  that  pattern  recognition  will  generalize  across  the  entire  threat 
category.  For  example,  screeners  are  typically  only  trained  on  a  small  sample  of  handguns.  It 
was  expected  that  specific  features  of  handguns  will  contribute  to  recognizing  all  members  of 
that  threat  classification,  regardless  of  the  profile  of  the  weapon. 
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Pattern  recognition  is  also  required  under  conditions  where  the  object  may  be  partially  obscured 
by  other  items  in  the  carry-on.  Pattern  recognition  under  such  adverse  conditions  had  to  be 
accomplished  from  only  partial  images.  Pattern  recognition  as  an  attribute  required  for 
successful  performance  in  security  screening  was  demonstrated  in  the  task  analysis.  Assessment 
of  pattern  recognition  capability  is  warranted  in  a  selection  battery  designed  to  hire  X-ray  security 
screeners.  However,  special  consideration  should  be  devoted  to  developing  instruments  that 
assess  pattern  recognition  under  conditions  where  only  partial  information  or  cues  are  available 
to  identify  an  object. 

Possible  tests  for  inclusion  should  assess  the  cognitive  factor  of  speed  of  closure.  The  Gestalt 
Completion  Test  and  the  Snowy  Pictures  Test  (Ekstrom,  French,  Harman,  and  Dermen,  1976) 
appear  on  the  surface  to  be  similar  to  the  tasks  involved  in  X-ray  screening.  That  is,  both 
instruments  measure  the  ability  to  detect  objects  that  are  embedded,  and  possibly  obscured  by  the 
background.  Past  research  (Crumley,  Pierce,  Schwalm,  Coke,  and  Brown,  1992)  has  indicated 
that  speed  of  closure  may  be  related  to  the  detection  of  targets  within  a  complex  visual  image. 
Instruments  that  assess  speed  of  closure  may  be  promising  for  predicting  performance  because  of 
their  apparent  similarity  to  the  X-ray  scanning  process. 

An  evaluation  of  the  relationship  between  field  dependence-independence  and  screening 
performance  should  also  be  explored.  Field  dependence-independence  is  a  measure  of  an 
individual’s  ability  to  disembed  figures  from  ground  relationships.  Although  much  controversy 
surrounds  the  nature  of  this  construct  since  first  introduced  by  Witkin  (1950),  there  is  some  face 
validity  in  applying  this  concept  to  screener  performance. 

The  task  of  the  X-ray  screener  is  after  all  disembedding  objects  (targets)  from  a  complex  visual 
field  (carry-on  items).  Screeners  are  required  to  identify  materials  from  images  under  conditions 
of  few  visual  cues  from  a  background  visual  field  that  may  obscure  or  mask  the  objects  being 
searched  for.  The  Embedded  Figures  Test  (EFT)  or  Group  Embedded  Figures  Test  (GEFT),  both 
developed  by  Witkin  (1971),  purports  to  assess  this  specific  ability.  These  instruments  may 
potentially  predict  screener  performance  and  warrant  an  investigation  of  their  predictive  validity. 

Whether  the  instruments  developed  by  Witkin  (1971)  assess  a  particular  personality  trait  or 
evaluate  a  unique  construct  related  to  spatial  ability  is  a  theoretical  issue  that  is  of  little 
importance  for  purposes  of  personnel  selection.  This  is  the  focus  of  the  current  controversy 
surrounding  field  dependence-independence.  Our  concern  is  the  relationship  of  this  construct  to 
success  as  an  X-ray  screener. 

Similar  tests  from  the  Kit  of  Factor-Referenced  Cognitive  Tests  (Ekstrom  et  al.,  1976)  measure 
the  cognitive  factor  of  flexibility  of  closure  (i.e..  Hidden  Figures  and  Hidden  Pattern  Tests)  and 
may  also  have  the  potential  as  predictive  instruments.  Both  tests  are  similar  to  the  EFT  and 
GEFT  in  that  they  provide  a  measure  of  the  ability  to  detect  specific  shapes  or  patterns  within  a 
complex  visual  field.  These  instruments  appear  related  to  the  perceptual  tasks  involved  in 
detecting  weapons  of  a  known  shape  or  pattern  in  a  carry-on  item.  These  marker  tests  for 
flexibility  of  closure  are  similar  to  X-ray  screening  as  they  require  identification  of  target  items  in 
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a  visual  field  by  comparison  with  mental  images.  An  examination  of  these  instruments’ 
relationships  to  X-ray  screening  performance  appears  worthwhile. 

In  summary,  the  results  from  the  JTA  appear  to  indicate  a  number  of  cognitive  and  perceptual 
abilities  that  may  be  related  to  successful  target  detection.  Specifically,  perceptual  abilities 
encompassing  the  constructs  of  flexibility  of  closure,  speed  of  closure,  perceptual  speed,  field 
dependence-independence,  and  pattern  recognition  may  contribute  to  the  acquisition  of  relevant 
information  while  scanning  X-ray  images.  Higher  level  cognitive  processes  involved  in  decision 
making  and  attentional  resources  encompassed  by  the  construct  of  vigilance  appear  to  be  required 
for  successful  screener  performance  as  well.  These  psychological  qualities  are  important  for  the 
role  they  play  in  analyzing  and  attending  to  the  relevant  input.  Together  with  the  assessment  of 
appropriate  personality  traits,  the  incorporation  of  perceptual  and  cognitive  assessment 
instruments  has  the  potential  to  establish  a  useful  personnel  selection  system. 

As  put  forth  earlier,  there  may  be  some  personality  traits  related  to  screener  success.  Several 
personality  traits  were  in  fact  referred  to  by  supervisory  and  management  personnel  during  the 
JTA  work  as  important  for  success  as  a  screener.  Personality  traits  were  often  referred  to  as  the 
“soft”  side  of  screener  requirements. 

The  JTA  was  not  oriented  toward  identifying  related  personality  traits.  This  is  not  the  purpose, 
nor  appropriate  methodology,  for  gathering  these  data.  However,  given  the  potential  utility  of 
assessing  personality  traits  for  predicting  performance,  and  some  of  the  earlier  findings  (ATA, 
1990)  that  indicate  possible  relationships,  these  issues  will  be  examined  during  the  next  phase  of 
the  investigation.  Delphi  small-group  workshops  with  screeners  and  supervisory  personnel 
provides  a  suitable  format  to  explore  these  concepts 
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APPENDIX  A 


JOB  TASK  ANALYSIS  FUNCTIONAL  FLOW  DIAGRAM 
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APPENDIX  B 


JOB  TASK  ANALYSIS  OBSERVATION/INTERVIEW  FORM 


JOB  TASK  ANALYSIS 


FACTORS  IN  THE  DECISION  TREE/ANALYSIS: 

Image  appears  on  screen  . . .  can  positive  identification  be  made  or  not? 

1)  No  threats  2)  Potential  threats  3)  Obvious  threats  4)  Hazmats/not  permissibles 

If  not . . .  decision  process  questions: 

1.  Is  the  size  large  enough  to  be  considered  further? 

2.  Is  the  pattem/shape/form  match  possible  problems? 

If  yes  ...  is  it  real  (gun)  or  not  (toy)  by  further  analysis? 

3.  Is  the  color  (shading)/impenetrability  indicative  of  further  analysis? 

4.  Are  components  (wires,  fluids,  batteries,  etc.)  present  to  warrant  further  analysis? 

5.  Can  known  or  recognized  objects  be  eliminated? 

6.  Is  any  portion  of  the  image  masked  by  opaqueness? 

7.  Can  the  object  be  required  to  undergo  further  analysis  if  seen  rotated  (compact),  profiled, 
or  bird’s  eye  view? 

8.  Is  depicted  angle  masking  identity? 

9.  Does  stimulus  match  known  library  of  targets?  (trained  images)  question  of 
generalizability? 

MISCELLANEOUS: 

Is  scan  pattern  in  totality,  individualized,  right  to  left,  etc.? 

ACTIONS: 

1 .  Re-enter  carry-on  through  X-ray  at  different  angle  and  repeat  decision  process 

2.  Hand  search 

3.  OPD  and  initiate  emergency  procedures 

4.  Pass 
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JOB  TASK  ANALYSIS 


VERBAL  PROBES: 

1.  Critical  cue  probes:  Verbally  probe  screener  as  to  those  cues  that  result  in  detailed 
examination.  Specifically  identify  characteristics.  Particular  attention  to  actions  where 
bag  check,  re-analysis,  or  belt  motion  is  halted.  Identify  features  (cues)  that  are  used. 

2.  Goal  alternate  decision/action  paths:  Determine  decision  and  action  paths  for  target 
identification.  Identify  specific  decision  points  and  flow  of  decision  logic. 

3.  Causal  factor  analysis:  Acquire  data  on  cues  used  that  are  not  present  in  the  image  (i.e., 
passenger  appearance,  passenger  flow  rate,  disposition  of  passenger,  etc.).  Identify 
factors  that  change/alter  the  nature  of  the  bag  scan/screen. 

4.  Hypothetical  probes:  Use  verbal  probes  to  phrase  “what  if’  scenarios.  Determine  and 
clarify  cues  and  characteristics  used  in  screening  images. 

5.  Decision  option  assessments:  Determine  cues  and  characteristics  that  result  in  differing 
decisions  and  actions.  Are  there  specific  cues  that  lead  to  specific  decisions  or  actions  in 
a  definable  trend? 

6.  Error  analysis:  Isolate  cues  or  elements  that  result  in  “false  alarms”  or  “misses.”  What 
did  the  screener  miss  that  resulted  in  an  error?  Were  “false  alarms”  provided 
performance  feedback,  i.e.,  physically  shown  object. 

7.  Direct  observation  with  cognitive  probes:  Ascertain,  by  using  structured  (data  recording 
form)  or  semi-structured  interviews,  those  cues  and  decision  points  the  screener  uses 
while  an  image  is  still  on  the  monitor.  Accomplish  during  training  sessions,  proficiency 
checks,  periods  of  slow  traffic,  during  bag  checks,  and  air  carrier  checks.  Use  test  items 
in  various  rotations  and  obscurity  to  present  an  image  on  the  monitor.  Ensure  screener 
physically  identifies  cues  in  response  to  cognitive  probes.  Determine  depth  of  screening 
examination  and  extent  of  the  number  of  cues. 
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PHASE  H:  SCREENER  (X-RAY  POSITION)  TASK  ANALYSIS 

1.  Query  the  nature  of  scan  (top-down,  center  outward,  right-left,  etc.);  i.e.,  technique  for 
viewing  image  or  movement  of  eyes  around  screen. 

2.  Importance  of,  or  degree  of  uncertainty  for  these  factors  (rated  simply  as  E,  H,  M,  L). 
This  is  to  determine  how  important  each  of  these  are  in  a  pass,  stop,  bag  check,  re-X-ray 
decision.  Whether  to  do  something  with  the  “bag”  or  not. 

a.  Size  of  questionable  area 

b.  Angle  object  appears.  Rotation  or  profile  of  certain  objects  creating  uncertainty  (side 
view,  top  view) 

c.  Shape  or  form  of  image  (outline) 

d.  Degree  of  clutter  of  image  (overpacked  carry-on) 

e.  Large  area  of  opaque  masking  contents  within  (can’t  see  objects  because  something 
blocks  view) 

f.  Are  questionable  items  rotated  mentally  to  make  decision 

g.  Shading,  impenetrability  of  X-ray,  or  color  of  object 

h.  Presence  of  possible  components  that  together  would  constitute  a  possible  threat  such  as 
wires,  batteries,  clocks,  timers,  bottles,  etc. 

I.  Computers,  printers,  stacks  of  paper 

j.  Flashlights 

k.  Guns  that  are  clearly  no  threat  (toy  guns),  plastic  handles,  etc. 

l.  Other  aspects  that  cause  uncertainty  or  concern  (itemize) 
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PHASE  ni:  SCREENER  TASK  ANALYSIS  (X-RAY  POSITION) 


1 .  Number  of  months  in  the  position:  _ 

2.  Determine  whether  or  not  the  following  factors  are  a  consideration  in  identifying  items. 
Simply  put,  are  these  factors  used  or  not  used  in  a  pass,  bag  check,  re-X-ray  decision. 
Whether  to  do  something  with  the  bag  or  not.  Positive  or  negative  response  is  all  that  is 
needed.  Frequency  or  routineness  of  using  factor  is  not  an  issue  here. 

a.  Size  of  questionable  area:  _ 

b.  Profile  or  angle  object  is  laying  in  carry-on:  _ 

i.e.,  Is  it  important  that  the  object  may  be  seen  from  the  side,  the  top  view,  or  at  an 
unusual  angle? 

b. 2.  Do  you  attempt  to  figure  out  what  the  object  may  be  by  trying  to  mentally  see  it  in  your 

^  head  from  different  viewpoints? 

We  are  trying  to  determine  if  screeners  mentally  rotate:  _ 

c.  Shape  or  form  of  the  image,  it’s  outline:  _ 

d.  The  amount  of  clutter  the  carry-on  appears:  _ 

In  other  words,  if  the  carry-on  is  especially  cluttered,  do  you  do  anything  different  or  special? 

e.  Large  area  of  shading  or  masking  which  blocks  view  of  contents  of  the  bag.  Can’t  quite 

determine  what’s  in  the  carry-on:  _ 

f.  The  impenetrability  of  the  X-ray,  i.e.,  dark  coloring  of  object: _ 

g.  Is  the  presence  of  possible  components  (i.e.,  wires,  batteries,  clocks,  timers,  bottles) 

considered  or  cause  for  bag  check:  _ 

3.  Do  any  of  the  following  items  factor  in  a  resulting  bag  check  or  closer  observation: 

*  Flashlights  : _ 

*  Computers  : _ 

*  Toy  Guns  : _ 

4.  Are  there  any  other  factors  or  items  that  cause  concern  or  you  look  for? 
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